WHAT IS CLAIMED IS: 

1. A computer-implemented method for populating an 
electronic form from an electronic image, the method 
comprising : 

(a) identifying a size, orientation and position 

of a first object having any arbitrary 
orientation within the electronic image; 

(b) identifying information elements from pixels 

within the electronic image that correspond 
to the first object; 

(c) displaying fields of the electronic form and 

the identified information elements to a 
user through a graphical user interface; 

(d) parsing the information elements into tagged 

groups of different information types; and 

(e) populating the fields of the electronic form 

with the tagged groups to produce a 
populated form and allowing the user to 
edit the populated fields through the 
graphical user interface. 

2. The method of claim 1 wherein (a) comprises 
identifying a size, orientation and position of the 
first object among a plurality of objects within the 
electronic image. 



3. The method of claim 1 wherein (a) comprises: 

classifying each pixel within the image to 
produce pixel classification data; 



defining an image function to process the pixel 

classification data; 
dividing the image into sub-images based on 

disparities in the image function; and 
processing the sub-images to determine a size, 

an orientation and a position for each of 

the objects, including the first object. 

4. The method of claim 3 and further comprising 
repeating the classifying pixels, defining an image 
function, and dividing of the image until the image 
contains a single object or the image can no longer 
be divided. 

5. The method of claim 3 wherein classifying 
comprises classifying each pixel as one of a data 
pixel or a background pixel. 

6. The method of claim 5 and further comprising: 
defining the image function as a sum of data 

pixels in a certain direction; 
calculating the image function in a first 

direction to generate a first data set; 
calculating the image function in a second 

direction to generate a second data set; 

and 

searching for disparities in the image function 
in the first direction and the image 
function in the second direction. 
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7. The method of claim 1 wherein: 

(b) comprises identifying text blocks and two- 
dimensional locations of the text blocks 
within the first object using optical 
character recognition; and 

(c) comprises displaying the identified text 
blocks to the user through the graphical 
user interface simultaneously with the 
fields of the electronic form. 

8. The method of claim 7 wherein: 

(c) comprises displaying the identified 
information elements within an image of the 
first object to the user through the 
graphical user interface; and 

(d) comprises highlighting the tagged groups of 

different information types within the 
image of the first object with visual 
indicators that indicate the different 
information types. 

9. The method of claim 1 wherein: 

(d) comprises receiving the information elements 

as untagged media from an untagged media 
store, and parsing the untagged media to 
identify information types of the 
information elements; 

(e) comprises automatically populating at least 

one of the fields with at least one of the 



information elements based on the 
information type of that element; and 
(c) comprises displaying the information 
elements through an object data graphical 
user interface and displaying the populated 
fields and any unpopulated fields through a 
form graphical user interface. 

10. The method of claim 9 wherein (c) further 
comprises employing visual indicator in the object 
data graphical user interface to indicate that an 
information element is compatible with a particular 
field in the form. 

11. The method of claim 9 wherein (c) further 
comprises indicating a status level associated with 
at least one information element in at least one 
populated field in the form. 

12. The method of claim 11 wherein (c) further 
comprises prompting the user to verify and/or correct 
the at least one information element in the at least 
one populated field based on the indicated status 
level . 

13. The method of cla im 12 wherein (d) comprises 
updating other fields of the populated form based on 
the user verification and/or correction of the at 
least one information element. 
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14. The method of claim 1 and further comprising: 

(f) writing side information gleaned from edits 

made to any of the populated fields to a 
side information store; and 

(g) re-parsing the information elements into 
tagged groups of different information 
types in (d) and re-populating at least one 
other field in (e) based on the side 
information. 

15. The method of claim 1 wherein the electronic 
image comprises a plurality of objects and the method 
further comprises : 

(f) performing (a) through (e) for each object, 
such that, for each object, fields of a 
corresponding electronic form are at least 
partially populated with information 
elements obtained from pixels within that 
object. 

16. The method of claim 15 wherein the plurality of 
objects comprises objects of different types. 

17. A computer-readable medium comprising computer- 
executable instructions that, when executed by a 
computer, performs a method comprising: 

(a) identifying a size, orientation and position 
of a first object having any arbitrary 
orientation within an electronic image; 



(b) identifying information elements from pixels 

within the electronic image that correspond 
to the first object; 

(c) displaying fields of the electronic form and 

the identified information elements to a 
user through a graphical user interface; 
and 

(d) parsing the information elements into tagged 

groups of different information types; 

(e) populating the fields of the electronic form 

with the tagged groups to produce a 
populated electronic form and allowing the 
user to edit the populated fields through 
the graphical user interface. 

18. The computer-readable medium of claim 17 wherein 
the electronic image comprises a plurality of 
objects, including the first object, and the method 
further comprises: 

(f) performing (a) through (e) for each object, 

such that, for each object, fields of a 
corresponding electronic form are at least 
partially populated with information 
elements obtained from pixels within that 
object . 



19. The computer-readable medium of claim 18 wherein 
the plurality of objects comprises objects of 
different types. 
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20. The computer-readable medium of claim 17 wherein 
(a) comprises: 

classifying each pixel within the image to 
produce pixel classification data; 

defining an image function to process the pixel 
classification data; 

dividing the image into sub-images based on 
disparities in the image function; and 

processing the sub-images to determine a size, 
an orientation and a position for each of 
the objects, including the first object. 

21. The computer-readable medium of claim 20 wherein 
classifying comprises classifying each pixel as one 
of a data pixel or a background pixel. 

22. The computer-readable medium of claim 21 and 
further comprising : 

defining the image function as a sum of data 

pixels in a certain direction; 
calculating the image function in a first 

direction to generate a first data set; 
calculating the image function in a second 

direction to generate a second data set; 

and 

searching for disparities in the image function 
in the first direction and the image 
function in the second direction. 
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23. The computer-readable medium of claim 17 
wherein : 

(b) comprises identifying text blocks and two- 
dimensional locations of the text blocks 
within the first object using optical 
character recognition; 

(c) comprises displaying the identified text 
blocks within an image of the first object 
to the user through the graphical user 
interface; and 

(d) comprises highlighting the tagged groups of 

different information types within the 
image of the first object with visual 
indicators that indicate the different 
information types. 



24. The computer-readable medium of claim 17 
wherein: 

(d) comprises receiving the information elements 

as untagged media from an untagged media 
store, and parsing the untagged media to 
identify information types of the 
information elements; 

(e) comprises automatically populating at least 

one of the fields with at least one of the 
information elements based on the 
information type of that element; and 
(c) comprises displaying the information 
elements through an object data graphical 
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user interface and displaying the populated 
fields and any unpopulated fields through a 
form graphical user interface. 

25. The computer-readable medium of claim 24 wherein 
(c) further comprises employing visual indicator in 
the object data graphical user interface to indicate 
that an information element is compatible with a 
particular field in the form. 

26. The computer-readable medium of claim 24 wherein 
(c) further comprises indicating a status level 
associated with at least one information element in 
at least one populated field in the form. 

27. The computer-readable medium of claim 26 wherein 

(c) further comprises prompting the user to verify 
and/or correct the at least one information element 
in the at least one populated field based on the 
indicated status level. 

28. The computer-readable medium of claim 27 wherein 

(d) comprises updating other fields of the populated 
form based on the user verification and/or correction 
of the at least one information element. 

29. The computer-readable medium of claim 17 and 
further comprising: 



(f) writing side information gleaned from edits 

made by the user in step (e) to a side 
information store; and 

(g) re-parsing the parsing the information 
elements into tagged groups of different 
information types in (d) and re-populating 
at least one other field in (e) based on 
the side information. 

30. A system for at least partially populating 

electronic forms, the system comprising: 

an object detection and extraction module, which 
processes pixels in the electronic image to 
identify a size, orientation and position 
of an object having any arbitrary 
orientation within the electronic image; 
an optical character recognition module, which 
identifies information elements from pixels 
within the electronic image that correspond 
to the first object; 
a graphical user interface, which simultaneously 
displays fields of the electronic form and 
the identified information elements to a 
user; and 

a parsing module, which parses the information 
elements into tagged groups of different 
information types and at least partially 
populates the fields with the tagged groups 
to produce a populated electronic form. 
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31. The system of claim 30 wherein the graphical 
user interface allows the user to edit the populated 
fields and any unpopulated fields. 

32. The system of claim 30 wherein the object 
detection and extraction module comprises: 

a data pixel detection module that classifies 
each pixel of the image and defines an 
image function; 

a segmentation module that is capable of 
dividing the image into smaller sub-images 
based on disparities in the image function; 
and 

a single object extraction module that processes 
the sub-images to detect a number, a size, 
an orientation and a position of objects 
within the image. 

33. The system of claim 32 wherein the data pixel 
detection module classifies each pixel as one of a 
data pixel or a background pixel. 

33. The system of claim 33 wherein the data pixel 
detection module defines the image function as a sum 
of data pixels in a certain direction, calculates the 
image function in a first direction to generate a 
first data set, calculates the image function in a 
second direction to generate a second data set, and 
searches for disparities in the image function in the 
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first direction and the image function in the second 
direction. 



34. The system of claim 30 wherein: 

the optical character recognition module 
comprises means for identifying text blocks 
and two-dimensional locations of the text 
blocks within the first object using 
optical character recognition; and 

the graphical user interface comprises means for 
displaying the identified text blocks 
within an image of the first object to the 
user through the graphical user interface 
and highlighting the tagged groups of 
different information types within the 
image of the first object with visual 
indicators that indicate the different 
information types. 

35. The system of claim 30 wherein: 

the parsing module comprises means for receiving 
the information elements as untagged media 
from an untagged media store, parsing the 
untagged media to identify information 
types of the information elements, and 
automatically populating at least one of 
the fields with at least one of the 
information elements based on the 
information type of that element; and 
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the graphical user interface comprises means for 
displaying the information elements through 
an object data graphical user interface and 
displaying the populated fields and any 
unpopulated fields through a form graphical 
user interface. 

36. The system of claim 35 wherein the object data 
graphical user interface comprises a visual indicator 
to indicate that an information element is compatible 
with a particular field in the form. 

37. The system of claim 35 wherein the form 
graphical user interface comprises a status level 
indicator associated with at least . one information 
element in at least one populated field in the form. 

38. The system of claim 37 wherein the graphical 
user interface comprises means for prompting the user 
to verify and/or correct at least one information 
element in at least one populated field based on the 
status level indicator associated with that field. 

39. The system of claim 31 wherein the parsing 
module comprises means for updating other fields of 
the populated form according to edits made by the 
user to the populated fields and the unpopulated 
fields . 



40. A method for populating electronic forms from an 
electronic image having first and second objects of 
different information types, the method comprising: 

identifying a size, orientation and position of 
the first and second objects within the 
electronic image and dividing the 
electronic image into sub-images 

corresponding to pixels in the electronic 
image associated with the size, orientation 
and position of each object; 

performing optical character recognition on each 
sub-image to identify untagged information 
elements within the corresponding object; 

for each sub-image, parsing the untagged 
information elements into tagged 

information elements ; 

populating fields in a first electronic form 
type with the tagged information elements 
identified from the sub-image of the first 
object to produce a first populated form; 

populating fields in a second electronic form 
type with the tagged information elements 
identified from the sub-image of the second 
object to produce a second populated form; 
and 

displaying the first and second populated forms 
and the untagged information elements to a 
user through a graphical user interface and 
allowing the user to edit the first and 
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second populated forms through the 
graphical user interface. 



41. The method of claim 40 wherein: 

the first object comprises a business card and 
the second object comprises a purchase 
receipt; and 

the first electronic form comprises a contact 
record of a software address book and the 
second electronic form comprises an 
electronic financial record of a software 
financial application . 



